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DETAILED ACTION 

1 . Applicant's arguments filed May 12, 2004 with respect to claims 1-23 have been 
considered but are moot in view of the applicant amendment necessitated new 
ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Rodriguez et al. (U.S. Patent No. 6,650,761 and Rodriguez hereinafter) in view of 
Purnaveja et al. (U.S. Patent No. 6,006,241 and Purnaveja hereinafter). 

As to claim 1 , Rodriguez teaches an encoder for encoding (col 43, lines 1 -67) a 
media file (i.e. MP3 file) (col 41 , lines 20-67) so that it is capable of being decoded (col 
43, lines 1-67) for playback (col 45, lines 1-67) as a media stream (col 43, lines 1-67), a 
method for encoding data (col 43, lines 1-67) and time restriction information (col 45, 
lines 1-25) with said media file (col 41, lines 20-67) such that when said encoded media 
file (col 41, lines 20-67) is decoded for playback as said media stream (col 43, lines 1- 
67), said data is reproduced in said media stream by said time restriction information 
(col 45, lines 1-25), comprising the steps of: (1) receiving a media file (col 41, lines 20- 
67); (2) receiving at least one at least one data segment (i.e. the watermark data) (col 
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42, lines 1-67); (3) receiving time restriction information (col 45, lines i-25) for each said 
data segment (col 43, lines 1-67) that specifies said time restriction (col 45, lines 1-25) 
when such that said data segment (col 43, lines 1-67) can be reproduced as part of said 
decoded (col 43, lines 1-67) media stream (col 43, lines 1-67) only if said time restriction 
(col 45, lines 1-25) corresponds to the current time and/or date (col 45, lines 1-25) when 
said media stream (col 43, lines 1-67) is being played back (col 45, lines 1-25); and (4) 
embedding (col 42, lines 1-67) as part of said encoded media file (col 41, lines 20-67) 
each said data segment (col 43, lines 1-67) and said corresponding time restriction 
information (col 45, lines 1-25). 

Rodriguez does not explicitly teach that the time restriction information comprises 
a time period predetermined and a data segment is contained in a data file. 

Purnaveja teaches that the time restriction information comprises a time period 

(i.e. "Another visual control aid, zoom bar 716, enables designer 219 to select the respective portions of the 
complete time tracks 750, 760, 770, 780, and 790, and defined by start time indicator 712 and end time indicator 

718, which is currently displayed by author tool 700. ") (col 7, lines 10-67) predetermined and a data 

Segment (i.e. the annotation data) (col 7, lines 10-67) is Contained in a data file (i.e. "When author 

module 318 has completed building an annotation stream, e.g., the flipper stream, the annotation stream is given a 
file name and loaded into a convenient server, e.g., stream server 220, for subsequent retrieval by client computer 

240.") (col 7, lines 10-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the time restriction information comprises a time period 
predetermined and a data segment is contained in a data file with the motivation to 
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synchronize the display of video streams with annotations, e.g., displayable events, 
such textual/graphical data in the form of HTML pages with java applets to be displayed 
in one or more event windows (Purnaveja, abstract). 



As to claim 2, Rodriguez does not explicitly teach that the time restriction 
information comprises a start time tag that specifies when said time period is to begin. 

Purnaveja teaches Purnaveja teaches that the time restriction information 
comprises a start time tag that specifies when said time period is to begin (i.e. "Another 

visual control aid, zoom bar 716, enables designer 219 to select the respective portions of the complete time tracks 
750, 760, 770, 780, and 790, and defined by start time indicator 712 and end time indicator 718, which is currently 

displayed by author tool 700 ") (col 7, lines 10-67) . 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the time restriction information comprises a start time tag that 
specifies when said time period is to begin with the motivation to synchronize the 
display of video streams with annotations, e.g., displayable events, such 
textual/graphical data in the form of HTML pages with java applets to be displayed in 
one or more event windows (Purnaveja, abstract). 

As to claim 3, Rodriguez does not explicitly teach that the time restriction 
information comprises a stop time tag that specifies when said time period is to end. 
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Purnaveja teaches Purnaveja teaches that the time restriction information 
comprises a stop time tag that specifies when said time period is to end (i.e. "Another visual 

control aid, zoom bar 716, enables designer 219 to select the respective portions of the complete time tracks 750, 
760, 770, 780, and 790, and defined by start time indicator 712 and end time indicator 718, which is currently 

displayed by author tool 700. ") (col 7, lines 10-67) . 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the time restriction information comprises a stop time tag that 
specifies when said time period is to end with the motivation to synchronize the display 
of video streams with annotations, e.g., displayable events, such textual/graphical data 
in the form of HTML pages with java applets to be displayed in one or more event 
windows (Purnaveja, abstract). 

As to claim 4, Rodriguez does not explicitly teach that the time restriction 
information comprises a start time tag that specifies when said time period is to begin 
and a stop time tag that specifies when said time period is to end. 

Purnaveja teaches Purnaveja teaches that the time restriction information 
comprises a start time tag that specifies when said time period is to begin and a stop 
time tag that specifies when said time period is to end (i.e. "Another visual control aid, zoom bar 

716, enables designer 219 to select the respective portions of the complete time tracks 750, 760, 770, 780, and 790, 
and defined by start time indicator 712 and end time indicator 718, which is currently displayed by author tool 

700/') (col 7, lines 10-67). 
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It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the time restriction information comprises a start time tag that 
specifies when said time period is to begin and a stop time tag that specifies when said 
time period is to end with the motivation to synchronize the display of video streams 
with annotations, e.g., displayable events, such textual/graphical data in the form of 
HTML pages with java applets to be displayed in one or more event windows 
(Purnaveja, abstract). 

As to claim 5, Rodriguez teaches that the embedding (col 42, lines 1-67) is such 
that said data is reproduced at a specified point in said media stream (col 43, lines 1-67; 
col 44, lines 1-67). 

As to claim 6, Rodriguez does not explicitly teach that the data segment 
comprises a document file. 

Purnaveja teaches that the data Segment (i.e. the annotation data) (col 7, lines 10-67) 

Comprises a document file (i.e. "When author module 318 has completed building an annotation stream, 
e.g., the flipper stream, the annotation stream is given a file name and loaded into a convenient server, e.g., stream 
server 220, for subsequent retrieval by client computer 240. ") (col 7, lines 10-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the data segment comprises a document file with the 
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motivation to synchronize the display of video streams with annotations, e.g., 
displayable events, such textual/graphical data in the form of HTML pages with java 
applets to be displayed in one or more event windows (Pumaveja, abstract). 

As to claim 7, Rodriguez does not teach that the data segment comprises an 
Internet address. 

Pumaveja teaches that the data segment comprises an Internet address (i.e. "The 

second type of annotation streams are locator annotation streams in which the displayable data is either too 
cumbersome and/or is continually evolving to be embedded as static data within the annotation stream. Instead, 
event locator(s) pointing to the location of the displayable data are stored in the annotation streams instead of the 
displayable data. Examples include URL addresses pointing to HTML pages, ") (col 6, lines 50-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Pumaveja to include that the data segment comprises an Internet address with the 
motivation to synchronize the display of video streams with annotations, e.g., 
displayable events, such textual/graphical data in the form of HTML pages with java 
applets to be displayed in one or more event windows (Pumaveja, abstract). 

As to claim 8, Rodriguez teaches that the media file is an audio file (col 41 , lines 
20-67) and said data segment (i.e. Watermark data) (col 42, lines 1-67) is a non-audio 
data segment (col 42, lines 1-67). 



Application/Control Number: 09/884,504 Page 8 

Art Unit: 2165 

As to claim 9, Rodriguez teaches that the audio file is an MP3 file (col 44, lines 1- 

67). 

As to claim 10, Rodriguez teaches an encoder (col 43, lines 1-67) for encoding 
(col 43, lines 1-67) a media file (i.e. MP3 file) (col 41 , lines 20-67) so that it is capable of 
being decoded (col 43, lines 1-67) for playback (col 45, lines 1-67) as a media stream 
(col 43, lines 1-67), a method for encoding (col 43, lines 1-67) a data (i.e. water 
marking data) (col 42, lines 1-67) and time restriction information (col 45, lines 1-25) for 
said segment with said media file (col 41 , lines 20-67) such that when said encoded 
media file (col 41 , lines 20-67) is decoded (col 43, lines 1-67) for playback (col 45, lines 
1-25) as said media stream, said data segment (col 42, lines 1-67) is reproduced 
according to time restriction information (col 45, lines 1-25), comprising the steps of: (a) 
receiving a media file; (b) receiving a segment of data (col 42, lines 1-67); (c) receiving 
the specifics of said time restriction (col 45, lines 1-25) when said data segment can be 
reproduced as part of said decoded (col 43, lines 1-67) media stream only if said time 
restriction (col 45, lines 1-25) corresponds to the current time and/or date (col 45, lines 
1-25) when said media stream is being played back (col 45, lines 1-25). 

Rodriguez does not explicitly teach that the data segment can be reproduced at a 
specified point in said media stream and only during a time period predetermined by 
time restriction information including one of the following: i) a start time, ii) a stop time, 
iii) both a start and a stop time. 
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Purnaveja teaches that the data segment (i.e. annotation data) can be 

reproduced at a Specified point (i.e. at the Start time) (i.e. "Another visual control aid, zoom bar 

716, enables designer 219 to select the respective portions of the complete time tracks 750, 760, 770, 780, and 790, 
and defined by start time indicator 712 and end time indicator 718, which is currently displayed by author tool 

700. ") (col 7, lines 10-67) in said media stream and only during a time period (i.e. from start 
time to end time) (col 7, lines io-67) predetermined by time restriction information (col 7, lines 
10-67) including one of the following: i) a start time, ii) a stop time, iii) both a start and a 

Stop time (col 7, lines 10-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the data segment can be reproduced at a specified point in 
said media stream and only during a time period predetermined by time restriction 
information including one of the following: i) a start time, ii) a stop time, iii) both a start 
and a stop time with the motivation to synchronize the display of video streams with 
annotations, e.g., displayable events, such textual/graphical data in the form of HTML 
pages with java applets to be displayed in one or more event windows (Purnaveja, 
abstract). 

As to claim 1 1 , Rodriguez teaches that the media file is an audio file (col 41 , lines 
20-67) and said data segment (i.e. Watermark data) (col 42, lines 1-67) is a non-audio 
data segment (col 42, lines 1-67). 
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As to claim 12, Rodriguez teaches a decoder for decoding (col 43, lines 1-67) an 
encoded (col 43, lines 1-67) media file (col 41 , lines 20-67) for playback (col 45, lines 1- 
67) as a media stream (col 43, lines 1-67), a method for decoding (col 43, lines 1-67) 
data and time restriction information (col 45, lines 1-25) from an encoded media file, 
comprising the steps of: (1 ) receiving a block (col 43, lines 1-67) of said encoded (col 
43, lines 1-67) media file (col 41, lines 20-67); (2) decoding (col 43, lines 1-67) a data 
segment embedded (col 42, lines 1-67) within said block (col 43, lines 1-67); (3) 
decoding (col 43, lines 1-67) said time restriction information (col 45, lines 1-25) 
embedded (col 42, lines 1-67) within said block for said data segment (col 43, lines 1- 
67), wherein said time restriction information (col 45, lines 1-25) specifies a time when 
said data segment is to be reproduced as part of said decoded (col 43, lines 1-67) 
media stream (col 43, lines 1-67); and (4) comparing said specified time (col 45, lines 1- 
25) to the current time of day and/or date when said media stream (col 43, lines 1-67) is 
being played back (col 45, lines 1-67); and (5) responsive to said comparison 
reproducing said data segment (col 43, lines 1-67) at a specified time (col 43, lines 1- 
67) for playback (col 45, lines 1-67) only during said specified time period (col 45, lines 
1-25). 

Rodriguez does not explicitly teach that the data segment can be reproduced at a 
specified point in said media stream and only during a time period predetermined by 
time restriction information. 

Purnaveja teaches that the data segment (i.e. annotation data) can be 
reproduced at a specified point (i.e. at the start time) (i.e. "Another visual control aid, zoom bar 
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716, enables designer 219 to select the respective portions of the complete time tracks 750, 760, 770, 780, and 790, 
and defined by start time indicator 712 and end time indicator 718, which is currently displayed by author tool 

700. ") (col 7, lines 10-67) in said media stream and only during a time period (i.e. from start 
time to end time) (col 7, lines io-67) predetermined by time restriction information (col 7, lines 

10-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the data segment can be reproduced at a specified point in 
said media stream and only during a time period predetermined by time restriction 
information with the motivation to synchronize the display of video streams with 
annotations, e.g., displayable events, such textual/graphical data in the form of HTML 
pages with java applets to be displayed in one or more event windows (Purnaveja, 
abstract). 

As to claim 13, Rodriguez does not explicitly teach that the reproducing of said 
data segment includes subsequent processing for enabling the data to be displayed to a 
user. 

Purnaveja teaches that the reproducing of said data segment (col 3, lines 1-25) 
includes subsequent processing for enabling the data to be displayed to a user (i.e. The 

second type of annotation streams are locator annotation streams in which the displayable data is either too 
cumbersome and/or is continually evolving to be embedded as static data within the annotation stream. Instead, 
event locator(s) pointing to the location of the displayable data are stored in the annotation streams instead of the 
displayable data. Examples include URL addresses pointing to HTML pages. " ... "In this example, URL addresses, 
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for synchronizing HTML page flips with video stream are provided to web browser 950 thereby permitting client 
computer 240 to subsequently retrieve and display various textual and graphical elements changing at 
predetermined points corresponding to the timeline of the video stream. Note that HTML pages can be retrieved 
from one or more web server(s) 230. ") (Examiner asserts that the client needs user input (or client has to determine 
if the user wants to reproduce the HTML pages in the web browser using the URL/ internet address provided in the 
annotation frames) to retrieve the HTML pages from the web server(s)) (col 6, lines 50-67; col 9, lines 45-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the reproducing of said data segment includes subsequent 
processing for enabling the data to be displayed to a user with the motivation to 
synchronize the display of video streams with annotations, e.g., displayable events, 
such textual/graphical data in the form of HTML pages with java applets to be displayed 
in one or more event windows (Purnaveja, abstract). 

As to claim 14, Rodriguez teaches a decoder for decoding an encoded media file 
for playback as a media stream, a method for decoding an embedded data segment 
and time restriction information from said encoded media file wherein said embedded 
data segment is to be reproduced at a specified point in said media stream only during 
a time period predetermined by said time restriction information, comprising the steps 
of: (a) reading a block of said encoded media file; (b) determining whether said 
embedded data segment exists within said block; (c) if said embedded data segment 
does not exist within said block, then proceeding to step (p); (d-n) determining if the 
embedded data segment specifies time restriction information; getting the current time 



Application/Control Number: 09/884,504 Page 13 

Art Unit: 2165 

of day and/or date when said media stream is being played back; if the time restriction 
information does not exist, then proceed to step (o); compare the time restriction 
information to the current time; if the current time is within time restriction information 
proceed to step (o); (o) reproducing said embedded segment; and (p) Continuing to 
step (a). 

Rodriguez does not explicitly teach that the data segment can be reproduced at a 
specified point in said media stream and only during a time period predetermined by 
time restriction information including one of the following: i) a start time tag, ii) a stop 
time tag, iii) both a start and a stop time tag. 

Purnaveja teaches that the data segment (i.e. annotation data) can be 

reproduced at a Specified point (i.e. at the Start time) (i.e. "Another visual control aid, zoom bar 

716, enables designer 219 to select the respective portions of the complete time tracks 750, 760, 770, 780, and 790, 
and defined by start time indicator 712 and end time indicator 718, which is currently displayed by author tool 

700. ") (col 7, lines 10-67) in said media stream and only during a time period (i.e. from start 
time to end time) (col 7, lines 10-67) predetermined by time restriction information (col 7, lines 
io-67) including one of the following: i) a start time tag, ii) a stop time tag, iii) both a start 
and a stop time tag (col 7, lines 10-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the data segment can be reproduced at a specified point in 
said media stream and only during a time period predetermined by time restriction 
information including one of the following: i) a start time tag, ii) a stop time tag, iii) both a 
start and a stop time tag with the motivation to synchronize the display of video streams 
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with annotations, e.g., displayable events, such textual/graphical data in the form of 
HTML pages with java applets to be displayed in one or more event windows 
(Purnaveja, abstract). 

As to claim 15, Rodriguez teaches that the media file is an audio file (col 41 , lines 
20-67) and said data segment (i.e. Watermark data) (col 42, lines 1-67) is a non-audio 
data segment (col 42, lines 1-67). 

As to claims 16 and 17, Rodriguez teaches the step of proceeding to step (o) if 

no Current time is available (i.e. if no real time clock is available then the date field can not be checked and 
the data block would be automatically reproduced and the next block would also be reproduced and so on.) (col 43, 
lines 1-67; col 45, lines 1-67). 

As to claim 18, As to claim 18, Rodriguez teaches reproducing embedded data 
segment (col 43, lines 1-67). 

Rodriguez does not teach that the step (o) further includes the step of 
determining whether said segment is reproduced based on input from a user. 

Purnaveja teaches that the step (o) further includes the step of determining 

Whether said Segment (i.e. the annotation frame) (Abstract; Fig.3; Fig.4B; Fig.8B; Fig.9; Fig.lOA) is 
reproduced based On input from a USer (i.e. "Flipper annotation frames are provided by stream 

server220 synchronously with the video/audio frames to the client module 960 so that the annotations, i.e., 
displayable events can be synchronized for display at client computer 24 (step 1060). In this example, URL 
addresses, for synchronizing HTML page flips with video stream are provided to web browser 950 thereby 
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permitting client computer 240 to subsequently retrieve and display various textual and graphical elements 
changing at predetermined points corresponding to the timeline of the video stream. Note that HTML pages can be 
retrieved from one or more web server(s) 230. ") (Examiner asserts that the client needs user input (or client has to 
determine if the user wants to reproduce the HTML pages in the web browser using the URL/ internet address 
provided in the annotation frames) to retrieve the HTML pages from the web server(s)) (col 7, lines 10-67; col 9, 

lines 45-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the step (o) further includes the step of determining whether 
said segment is reproduced based on input from a user with the motivation to 
synchronize the display of video streams with annotations, e.g., displayable events, 
such textual/graphical data in the form of HTML pages with java applets to be displayed 
in one or more event windows (Purnaveja, abstract). 

As to claim 19, Rodriguez teaches reproducing embedded data segment (col 43, 
lines 1-67). 

Rodriguez does not teach that the reproducing of said data segment in step (o) 
includes subsequent processing for enabling the data to be displayed to a user. 

Purnaveja teaches the reproducing of said data segment (i.e. interpreting the encoded 

annotated frame by annotation interpreter 963) (Abstract; Fig.3; Fig.4B; Fig.8B; Fig.9; Fig.lOA; col 3, lines 1-25; 

col 9, lines 10-25) in step (o) includes subsequent processing for enabling the data to be 

displayed tO a USer (i.e. The second type of annotation streams are locator annotation streams in which the 

displayable data is either too cumbersome and/or is continually evolving to be embedded as static data within the 
annotation stream. Instead, event locator(s) pointing to the location of the displayable data are stored in the 
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annotation streams instead of the displayable data. Examples include URL addresses pointing to HTML pages. " ... 
"In this example, URL addresses, for synchronizing HTML page flips with video stream are provided to web 
browser 950 thereby permitting client computer 240 to subsequently retrieve and display various textual and 
graphical elements changing at predetermined points corresponding to the timeline of the video stream. Note that 
HTML pages can be retrieved from one or more web server(s) 230. ") (Examiner asserts that the client needs user 
input (or client has to determine if the user wants to reproduce the HTML pages in the web browser using the URL/ 
internet address provided in the annotation frames) to retrieve the HTML pages from the web server(s)) (col 6, lines 

50-67; col 9, lines 45-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the reproducing of said data segment in step (o) includes 
subsequent processing for enabling the data to be displayed to a user with the 
motivation to synchronize the display of video streams with annotations, e.g., 
displayable events, such textual/graphical data in the form of HTML pages with java 
applets to be displayed in one or more event windows (Purnaveja, abstract). 

As to claim 20, Rodriguez teaches embedding data segment with the media data 
(col 42, lines 1-67). 

Rodriguez does not teach that the data segment comprises an Internet address. 
Purnaveja teaches that the data segment comprises an Internet address (i.e. 'The 

second type of annotation streams are locator annotation streams in which the displayable data is either too 
cumbersome and/or is continually evolving to be embedded as static data within the annotation stream. Instead, 
event locator(s) pointing to the location of the displayable data are stored in the annotation streams instead of the 

displayable data. Examples include URL addresses pointing to HTML pages. ") (col 6, lines 50-67). 
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It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 

i 

Purnaveja to include that the data segment comprises an Internet address with the 
motivation to synchronize the display of video streams with annotations, e.g., 
displayable events, such textual/graphical data in the form of HTML pages with java 
applets to be displayed in one or more event windows (Purnaveja, abstract). 

As to claim 21 , Rodriguez does not teach that processing includes linking to an 
internet document determined by said Internet address. 

Purnaveja teaches that the processing includes linking to an Internet document 

(i.e. HTML pages) (col 6, lines 49-67) determined by Said Internet address (i.e. "Examples include URL 
addresses pointing to HTML pages. ") (col 6, lines 49-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that processing includes linking to an internet document 
determined by said Internet address with the motivation to synchronize the display of 
video streams with annotations, e.g., displayable events, such textual/graphical data in 
the form of HTML pages with java applets to be displayed in one or more event 
windows (Purnaveja, abstract). 

As to claim 22, Rodriguez teaches a system for encoding (col 43, lines 1-67) data 
and time restriction information (col 45, lines 1-25) with a media file (i.e. MP3 file) (col 
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41 , lines 20-67) and for the later decoding of said encoded (col 43, lines 1-67) media file 
such that when said encoded (col 43, lines 1-67) media file (col 41 , lines 20-67) is 
decoded for playback (col 45, lines 1-67) as a media stream (col 43, lines 1-67), said 
data (i.e. the watermark data) (col 42, lines 1-67) is reproduced in said media stream 
only during a time predetermined by said time restriction information (col 45, lines 1-25), 
comprising: an encoder (col 43, lines 1-67) operative to receive said media file (col 41, 
lines 20-67), at least one data segment (i.e. the watermark data) (col 42, lines 1-67), 
and time restriction information (col 45, lines 1-25) for each said data segment (i.e. the 
watermark data) (col 42, lines 1-67) and operative to embed as part of said media file 
said data segment (col 42, lines 1-67) and said corresponding time restriction 
information (col 45, lines 1-25); and a decoder (col 43, lines 1-67) operative to separate 
said encoded (col 43, lines 1-67) media file (col 41, lines 20-67) into said decoded (col 
43, lines 1-67) media stream (col 43, lines 1-67), said data segment and said time 
restriction information (col 45, lines 1-25), operative to compare said specified time to 
the current time of day and/or date (col 45, lines 1-25) when said media stream (col 43, 
lines 1-67) is being played back and operative in response to said comparison to 
playback (col 45, lines 1-67) said data segment as part of said decoded (col 43, lines 1- 
67) media stream only during said time specified by said corresponding time restriction 
information (col 45, lines 1-25). . 

Rodriguez does not explicitly teach that the time restriction information comprises 
a time period predetermined and a data segment is contained in a data file. 
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Purnaveja teaches that the time restriction information comprises a time period 

(i.e. "Another visual control aid, zoom bar 716, enables designer 219 to select the respective portions of the 
complete time tracks 750, 760, 770, 780, and 790, and defined by start time indicator 712 and end time indicator 
718, which is currently displayed by author tool 700. ") (col 7, lines 10-67) predetermined and a data 
Segment (i.e. the annotation data) (col 7, lines 10-67) is contained in a data file (i.e. "When author 
module 318 has completed building an annotation stream, e.g., the flipper stream, the annotation stream is given a 
file name and loaded into a convenient server, e.g., stream server 220, for subsequent retrieval by client computer 

240. ") (col 7, lines 10-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the time restriction information comprises a time period 
predetermined and a data segment is contained in a data file with the motivation to 
synchronize the display of video streams with annotations, e.g., displayable events, 
such textual/graphical data in the form of HTML pages with java applets to be displayed 
in one or more event windows (Purnaveja, abstract). 

As to claim 23, Rodriguez teaches a method for encoding (col 43, lines 1-67) and 
thereafter decoding (col 43, lines 1-67) a media file (i.e. MP3 file) (col 41, lines 20-67) 
such that a data segment (i.e. the watermark data) (col 42, lines 1-67) embedded in 
said media file (col 41 , lines 20-67) after encoding is reproduced during decoding (col 
43, lines 1-67) at a specified point during a predetermined time period (col 45, lines 1- 
25) for playback (col 45, lines 1-67) as a media stream (col 43, lines 1-67) for a user, 
comprising the steps of: (1) receiving a media file (col 41, lines 20-67); (2) receiving at 
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least one data file comprising at least one data segment (i.e. the watermark data) (col 
42, lines 1-67); (3) receiving time restriction information (col 45, lines 1-25) for each said 
data segment (i.e. the watermark data) (col 42, lines 1-67) that specifies said time 
period (col 45, lines 1 -25) when said data segment is to be reproduced as part of said 
decoded (col 43, lines 1-67) media stream; and (4) embedding as part of said encoded 
media file each said data segment (col 42, lines 1-67) and said corresponding time 
restriction information (col 45, lines 1-25) such that said data segment (col 42, lines 1- 
67) is reproduced by a decoder (col 43, lines 1-67) during said specified time period; (5) 
receiving a block of said encoded media file (col 41, lines 20-67); (6) decoding (col 43, 
lines 1-67) said data segment (col 42, lines 1-67) embedded within said block; (7) 
decoding (col 43, lines 1-67) said time restriction information (col 45, lines 1-25) 
embedded within said block for said data segment (col 42, lines 1-67), wherein said 
time restriction information (col 45, lines 1-25) specifies a time period when said data 
segment is to be reproduced as part of said decoded (col 43, lines 1-67) media stream; 
and (8) comparing said specified time period (col 45, lines 1-25) to the current time of 
day and/or date (col 45, lines 1-25) when said media stream (col 43, lines 1-67) is being 
played back', and (9) responsive to said comparison, reproducing said data segment 
(col 42, lines 1-67) at a specified point in said decoded (col 43, lines 1-67) media stream 
for playback (col 45, lines 1-67) only during said specified time period (col 45, lines 1- 
25). 

Rodriguez does not explicitly teach that the time restriction information comprises 
a time period predetermined and a data segment is contained in a data file. 
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Purnaveja teaches that the time restriction information comprises a time period 

(i.e. "Another visual control aid, zoom bar 716, enables designer 219 to select the respective portions of the 
complete time tracks 750, 760, 770, 780, and 790, and defined by start time indicator 712 and end time indicator 
718, which is currently displayed by author tool 700. ") (col 7, lines 10-67) predetermined and a data 
Segment (i.e. the annotation data) (col 7, lines 10-67) is Contained in a data file (i.e. "When author 
module 318 has completed building an annotation stream, e.g, the flipper stream, the annotation stream is given a 
file name and loaded into a convenient server, e.g., stream server 220, for subsequent retrieval by client computer 

240. ") (col 7, lines 10-67). 

It would have been obvious to a person of ordinary skill in the art at the time of 
Applicant's invention to modify the teachings of Rodriguez with the teachings of 
Purnaveja to include that the time restriction information comprises a time period 
predetermined and a data segment is contained in a data file with the motivation to 
synchronize the display of video streams with annotations, e.g., displayable events, 
such textual/graphical data in the form of HTML pages with java applets to be displayed 
in one or more event windows (Purnaveja, abstract). 

Points of Contact 

3. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Apu M. Mofiz whose telephone number is (571 ) 272- 
4080. The examiner can normally be reached on Monday - Thursday 8:00 A.M. to 4:30 
P.M. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dov Popovici can be reached at (571 ) 272-4083. The fax numbers for the 
group is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application should 
be directed to the Group receptionist whose telephone number is (703) 305-9600. 
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